Goal-oriented inference of environment from redundant observations.

The agent learns to organize decision behavior to achieve a behavioral goal, such as reward maximization, and reinforcement learning is often used for this optimization. Learning an optimal behavioral strategy is difficult under the uncertainty that events necessary for learning are only partially observable, called as Partially Observable Markov Decision Process (POMDP). However, the real-world environment also gives many events irrelevant to reward delivery and an optimal behavioral strategy. The conventional methods in POMDP, which attempt to infer transition rules among the entire observations, including irrelevant states, are ineffective in such an environment. Supposing Redundantly Observable Markov Decision Process (ROMDP), here we propose a method for goal-oriented reinforcement learning to efficiently learn state transition rules among reward-related "core states" from redundant observations. Starting with a small number of initial core states, our model gradually adds new core states to the transition diagram until it achieves an optimal behavioral strategy consistent with the Bellman equation. We demonstrate that the resultant inference model outperforms the conventional method for POMDP. We emphasize that our model only containing the core states has high explainability. Furthermore, the proposed method suits online learning as it suppresses memory consumption and improves learning speed.

Distinct Mechanisms of Over-Representation of Landmarks and Rewards in the Hippocampus.

In the hippocampus, locations associated with salient features are represented by a disproportionately large number of neurons, but the cellular and molecular mechanisms underlying this over-representation remain elusive. Using longitudinal calcium imaging in mice learning to navigate in virtual reality, we find that the over-representation of reward and landmark locations are mediated by persistent and separable subsets of neurons, with distinct time courses of emergence and differing underlying molecular mechanisms. Strikingly, we find that in mice lacking Shank2, an autism spectrum disorder (ASD)-linked gene encoding an excitatory postsynaptic scaffold protein, the learning-induced over-representation of landmarks was absent whereas the over-representation of rewards was substantially increased, as was goal-directed behavior. These findings demonstrate that multiple hippocampal coding processes for unique types of salient features are distinguished by a Shank2-dependent mechanism and suggest that abnormally distorted hippocampal salience mapping may underlie cognitive and behavioral abnormalities in a subset of ASDs.

Orchestrated ensemble activities constitute a hippocampal memory engram.

The brain stores and recalls memories through a set of neurons, termed engram cells. However, it is unclear how these cells are organized to constitute a corresponding memory trace. We established a unique imaging system that combines Ca2+ imaging and engram identification to extract the characteristics of engram activity by visualizing and discriminating between engram and non-engram cells. Here, we show that engram cells detected in the hippocampus display higher repetitive activity than non-engram cells during novel context learning. The total activity pattern of the engram cells during learning is stable across post-learning memory processing. Within a single engram population, we detected several sub-ensembles composed of neurons collectively activated during learning. Some sub-ensembles preferentially reappear during post-learning sleep, and these replayed sub-ensembles are more likely to be reactivated during retrieval. These results indicate that sub-ensembles represent distinct pieces of information, which are then orchestrated to constitute an entire memory.

Effects of Voice Therapy on Laryngeal Motor Units During Phonation in Chronic Superior Laryngeal Nerve Paresis Dysphonia.

OBJECTIVES: Injury to the superior laryngeal nerve can result in dysphonia, and in particular, loss of vocal range. It can be an especially difficult problem to address with either voice therapy or surgical intervention. Some clinicians and scientists suggest that combining vocal exercises with adjunctive neuromuscular electrical stimulation may enhance the positive effects of voice therapy for superior laryngeal nerve paresis (SLNP). However, the effects of voice therapy without neuromuscular electrical stimulation are unknown. The purpose of this retrospective study was to demonstrate the clinical effectiveness of voice therapy for rehabilitating chronic SLNP dysphonia in two subjects, using interspike interval (ISI) variability of laryngeal motor units by laryngeal electromyography (LEMG). METHODS: Both patients underwent LEMG and were diagnosed with having 70% recruitment of the cricothyroid muscle, and 70% recruitment of the cricothyroid and thyroarytenoid muscles, respectively. Both patients received voice therapy for 3 months. Grade, roughness, breathiness, asthenia, and strain (GRBAS) scale, stroboscopic examination, aerodynamic assessment, acoustic analysis, and Voice Handicap Index-10 were performed before and after voice therapy. Mean ISI variability during steady phonation was also assessed. RESULTS: After voice therapy, both patients showed improvement in vocal assessments by acoustic, aerodynamic, GRBAS, and Voice Handicap Index-10 analysis. LEMG indicated shortened ISIs in both cases. CONCLUSIONS: This study suggests that voice therapy for chronic SLNP dysphonia can be useful for improving SLNP and voice quality.

Medial Frontal Circuit Dynamics Represents Probabilistic Choices for Unfamiliar Sensory Experience.

Neurons in medial frontal cortex (MFC) receive sensory signals that are crucial for decision-making behavior. While decision-making is easy for familiar sensory signals, it becomes more elaborative when sensory signals are less familiar to animals. It remains unclear how the population of neurons enables the coordinate transformation of such a sensory input into ambiguous choice responses. Furthermore, whether and how cortical oscillations temporally coordinate neuronal firing during this transformation has not been extensively studied. Here, we recorded neuronal population responses to familiar or unfamiliar auditory cues in rat MFC and computed their probabilistic evolution. Population responses to familiar sounds organize into neuronal trajectories containing multiplexed sensory, motor, and choice information. Unfamiliar sounds, in contrast, evoke trajectories that travel under the guidance of familiar paths and eventually diverge to unique decision states. Local field potentials exhibited beta- (15-20 Hz) and gamma-band (50-60 Hz) oscillations to which neuronal firing showed modest phase locking. Interestingly, gamma oscillation, but not beta oscillation, increased its power abruptly at some timepoint by which neural trajectories for different choices were near maximally separated. Our results emphasize the importance of the evolution of neural trajectories in rapid probabilistic decisions that utilize unfamiliar sensory information.

Spike detection from noisy neural data in linear-probe recordings.

Simultaneous recordings of multiple neuron activities with multi-channel extracellular electrodes are widely used for studying information processing by the brain's neural circuits. In this method, the recorded signals containing the spike events of a number of adjacent or distant neurons must be correctly sorted into spike trains of individual neurons, and a variety of methods have been proposed for this spike sorting. However, spike sorting is computationally difficult because the recorded signals are often contaminated by biological noise. Here, we propose a novel method for spike detection, which is the first stage of spike sorting and hence crucially determines overall sorting performance. Our method utilizes a model of extracellular recording data that takes into account variations in spike waveforms, such as the widths and amplitudes of spikes, by detecting the peaks of band-pass-filtered data. We show that the new method significantly improves the cost-performance of multi-channel electrode recordings by increasing the number of cleanly sorted neurons.

Reward-modulated motor information in identified striatum neurons.

It is widely accepted that dorsal striatum neurons participate in either the direct pathway (expressing dopamine D1 receptors) or the indirect pathway (expressing D2 receptors), controlling voluntary movements in an antagonistically balancing manner. The D1- and D2-expressing neurons are activated and inactivated, respectively, by dopamine released from substantia nigra neurons encoding reward expectation. However, little is known about the functional representation of motor information and its reward modulation in individual striatal neurons constituting the two pathways. In this study, we juxtacellularly recorded the spike activity of single neurons in the dorsolateral striatum of rats performing voluntary forelimb movement in a reward-predictable condition. Some of these neurons were identified morphologically by a combination of juxtacellular visualization and in situ hybridization for D1 mRNA. We found that the striatal neurons exhibited distinct functional activations before and during the forelimb movement, regardless of the expression of D1 mRNA. They were often positively, but rarely negatively, modulated by expecting a reward for the correct motor response. The positive reward modulation was independent of behavioral differences in motor performance. In contrast, regular-spiking and fast-spiking neurons in any layers of the motor cortex displayed only minor and unbiased reward modulation of their functional activation in relation to the execution of forelimb movement. Our results suggest that the direct and indirect pathway neurons cooperatively rather than antagonistically contribute to spatiotemporal control of voluntary movements, and that motor information is subcortically integrated with reward information through dopaminergic and other signals in the skeletomotor loop of the basal ganglia.

The synchronous activity of lateral habenular neurons is essential for regulating hippocampal theta oscillation.

Lateral habenula (LHb) has attracted growing interest as a regulator of serotonergic and dopaminergic neurons in the CNS. However, it remains unclear how the LHb modulates brain states in animals. To identify the neural substrates that are under the influence of LHb regulation, we examined the effects of rat LHb lesions on the hippocampal oscillatory activity associated with the transition of brain states. Our results showed that the LHb lesion shortened the theta activity duration both in anesthetized and sleeping rats. Furthermore, this inhibitory effect of LHb lesion on theta maintenance depended upon an intact serotonergic median raphe, suggesting that LHb activity plays an essential role in maintaining hippocampal theta oscillation via the serotonergic raphe. Multiunit recording of sleeping rats further revealed that firing of LHb neurons showed significant phase-locking activity at each theta oscillation cycle in the hippocampus. LHb neurons showing activity that was coordinated with that of the hippocampal theta were localized in the medial LHb division, which receives afferents from the diagonal band of Broca (DBB), a pacemaker region for the hippocampal theta oscillation. Thus, our findings indicate that the DBB may pace not only the hippocampus, but also the LHb, during rapid eye movement sleep. Since serotonin is known to negatively regulate theta oscillation in the hippocampus, phase-locking activity of the LHb neurons may act, under the influence of the DBB, to maintain the hippocampal theta oscillation by modulating the activity of serotonergic neurons.

Spike sorting of heterogeneous neuron types by multimodality-weighted PCA and explicit robust variational Bayes.

This study introduces a new spike sorting method that classifies spike waveforms from multiunit recordings into spike trains of individual neurons. In particular, we develop a method to sort a spike mixture generated by a heterogeneous neural population. Such a spike sorting has a significant practical value, but was previously difficult. The method combines a feature extraction method, which we may term "multimodality-weighted principal component analysis" (mPCA), and a clustering method by variational Bayes for Student's t mixture model (SVB). The performance of the proposed method was compared with that of other conventional methods for simulated and experimental data sets. We found that the mPCA efficiently extracts highly informative features as clusters clearly separable in a relatively low-dimensional feature space. The SVB was implemented explicitly without relying on Maximum-A-Posterior (MAP) inference for the "degree of freedom" parameters. The explicit SVB is faster than the conventional SVB derived with MAP inference and works more reliably over various data sets that include spiking patterns difficult to sort. For instance, spikes of a single bursting neuron may be separated incorrectly into multiple clusters, whereas those of a sparsely firing neuron tend to be merged into clusters for other neurons. Our method showed significantly improved performance in spike sorting of these "difficult" neurons. A parallelized implementation of the proposed algorithm (EToS version 3) is available as open-source code at http://etos.sourceforge.net/.

Accurate spike sorting for multi-unit recordings.

Simultaneous recordings with multi-channel electrodes are widely used for studying how multiple neurons are recruited for information processing. The recorded signals contain the spike events of a number of adjacent or distant neurons and must be sorted correctly into spike trains of individual neurons. Several mathematical methods have been proposed for spike sorting but the process is difficult in practice, as extracellularly recorded signals are corrupted by biological noise. Moreover, spike sorting is often time-consuming, as it usually requires corrections by human operators. Methods are needed to obtain reliable spike clusters without heavy manual operation. Here, we introduce several methods of spike sorting and compare the accuracy and robustness of their performance by using publicized data of simultaneous extracellular and intracellular recordings of neuronal activity. The best and excellent performance was obtained when a newly proposed filter for spike detection was combined with the wavelet transform and variational Bayes for a finite mixture of Student's t-distributions, namely, robust variational Bayes. Wavelet transform extracts features that are characteristic of the detected spike waveforms and the robust variational Bayes categorizes the extracted features into clusters corresponding to spikes of the individual neurons. The use of Student's t-distributions makes this categorization robust against noisy data points. Some other new methods also exhibited reasonably good performance. We implemented all of the proposed methods in a C++ code named 'EToS' (Efficient Technology of Spike sorting), which is freely available on the Internet.

Microcircuitry coordination of cortical motor information in self-initiation of voluntary movements.

Motor cortex neurons are activated at different times during self-initiated voluntary movement. However, the manner in which excitatory and inhibitory neurons in distinct cortical layers help to organize voluntary movement is poorly understood. We carried out juxtacellular and multiunit recordings from actively behaving rats and found temporally and functionally distinct activations of excitatory pyramidal cells and inhibitory fast-spiking interneurons. Across cortical layers, pyramidal cells were activated diversely for sequential motor phases (for example, preparation, initiation and execution). In contrast, fast-spiking interneurons, including parvalbumin-positive basket cells, were recruited predominantly for motor execution, with pyramidal cells producing a command-like activity. Thus, fast-spiking interneurons may underlie command shaping by balanced inhibition or recurrent inhibition, rather than command gating by temporally alternating excitation and inhibition. Furthermore, initiation-associated pyramidal cells excited similar and different functional classes of neurons through putative monosynaptic connections. This suggests that these cells may temporally integrate information to initiate and coordinate voluntary movement.

Synchronous and asynchronous bursting states: role of intrinsic neural dynamics.

Brain signals such as local field potentials often display gamma-band oscillations (30-70 Hz) in a variety of cognitive tasks. These oscillatory activities possibly reflect synchronization of cell assemblies that are engaged in a cognitive function. A type of pyramidal neurons, i.e., chattering neurons, show fast rhythmic bursting (FRB) in the gamma frequency range, and may play an active role in generating the gamma-band oscillations in the cerebral cortex. Our previous phase response analyses have revealed that the synchronization between the coupled bursting neurons significantly depends on the bursting mode that is defined as the number of spikes in each burst. Namely, a network of neurons bursting through a Ca(2+)-dependent mechanism exhibited sharp transitions between synchronous and asynchronous firing states when the neurons exchanged the bursting mode between singlet, doublet and so on. However, whether a broad class of bursting neuron models commonly show such a network behavior remains unclear. Here, we analyze the mechanism underlying this network behavior using a mathematically tractable neuron model. Then we extend our results to a multi-compartment version of the NaP current-based neuron model and prove a similar tight relationship between the bursting mode changes and the network state changes in this model. Thus, the synchronization behavior couples tightly to the bursting mode in a wide class of networks of bursting neurons.

Difference and similarity between non-alcoholic steatohepatitis and alcoholic liver disease.

BACKGROUND: Non-alcoholic steatohepatitis (NASH) and alcoholic liver disease (ALD) are extremely similar in the pathologic findings and pathogenesis. This study aimed to elucidate the difference and similarity between these diseases. METHODS: Twenty-six patients with NASH and 26 with ALD including 11 with alcoholic hepatitis underwent clinico-pathologic analysis. The visceral fat area and liver/spleen ratio, an index of the hepatic fat content, were evaluated with computed tomography. The hepatic iron deposit and oxidative stress induced-lipid peroxidation were estimated by Prussian blue staining and 3-nitrotyrosine staining, respectively. RESULTS: The most prominent difference between NASH and ALD was the nutritional status, although elevation of AST/ALT ratio and gamma-GT is relatively characteristic of ALD. NASH was more frequently associated with diabetes mellitus as compared with ALD. The BMI and serum levels of total cholesterol and cholinesterase were higher in NASH than in ALD. Although the degree and distribution of fibrosis and necro-inflammatory reaction were similar in NASH and ALD, steatosis was more severe in NASH than in ALD. The liver/spleen ratio was lower and the visceral fat area was larger in NASH than in ALD, regardless of the coincidence of alcoholic hepatitis. Interestingly, the visceral fat area positively correlated with ALT and HOMA-IR in NASH, whereas these correlations were not observed in ALD. The hepatic iron deposit was less in NASH than in ALD, whereas lipid peroxidation in NASH was similar to that in ALD with alcoholic hepatitis and more advanced as compared with that in ALD without alcoholic hepatitis. CONCLUSIONS: NASH was characterized with over-nutrition and visceral fat type obesity as compared with ALD. The visceral fat accumulation was associated with hepatic inflammation and insulin resistance in NASH, but not in ALD. The difference in the nutritional status between NASH and ALD is not only reflected in the clinical features but also may closely associate with the mechanisms of hepatocellular damage in these diseases.

Gamma rhythmic bursts: coherence control in networks of cortical pyramidal neurons.

Much evidence indicates that synchronized gamma-frequency (20-70 Hz) oscillation plays a significant functional role in the neocortex and hippocampus. Chattering neuron is a possible neocortical pacemaker for the gamma oscillation. Based on our recent model of chattering neurons, here we study how gamma-frequency bursting is synchronized in a network of these neurons. Using a phase oscillator description, we first examine how two coupled chattering neurons are synchronized. The analysis reveals that an incremental change of the bursting mode, such as from singlet to doublet, always accompanies a rapid transition from antisynchronous to synchronous firing. The state transition occurs regardless of what changes the bursting mode. Within each bursting mode, the neuronal activity undergoes a gradual change from synchrony to antisynchrony. Since the sensitivity to Ca(2+) and the maximum conductance of Ca(2+)-dependent cationic current as well as the intensity of input current systematically control the bursting mode, these quantities may be crucial for the regulation of the coherence of local cortical activity. Numerical simulations demonstrate that the modulations of the calcium sensitivity and the amplitude of the cationic current can induce rapid transitions between synchrony and asynchrony in a large-scale network of chattering neurons. The rapid synchronization of chattering neurons is shown to synchronize the activities of regular spiking pyramidal neurons at the gamma frequencies, as may be necessary for selective attention or binding processing in object recognition.